Interconnected devices, such as networked sensors and computing devices, can generate and send data pertinent to various operations, transactions, or other events to remote devices for storage and further analysis. For example, mobile phones can generate data indicating their locations, the applications being used by the phone users, and so on, at least some of which can be collected and analyzed in order to present customized coupons, advertisements and the like to the users. The analysis of data collected by surveillance cameras may be useful in preventing and/or solving crimes, and data collected from sensors embedded at various location within airplane engines, automobiles or complex machinery may be used for various purposes such as preventive maintenance, improving efficiency and lowering costs.
Large amounts of such data may be generated and sent in streaming fashion so that the data can be captured in real time. However, over time the volume of streaming data may prove difficult to efficiently analyze. Different data formats for data records within data streams could require different operations to be performed in order to interpret or manipulate the data, making it difficult to utilize a common schema to handle multiple data streams. The volume of data within a data stream may change over time, complicating efforts to acquire the appropriate number of resources for efficiently analyzing the data stream.
While embodiments are described herein by way of example for several embodiments and illustrative drawings, those skilled in the art will recognize that embodiments are not limited to the embodiments or drawings described. It should be understood, that the drawings and detailed description thereto are not intended to limit embodiments to the particular form disclosed, but on the contrary, the intention is to cover all modifications, equivalents and alternatives falling within the spirit and scope as defined by the appended claims. The headings used herein are for organizational purposes only and are not meant to be used to limit the scope of the description or the claims. As used throughout this application, the word “may” is used in a permissive sense (i.e., meaning having the potential to), rather than the mandatory sense (i.e., meaning must). Similarly, the words “include,” “including,” and “includes” mean including, but not limited to.